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This booklet uses materials developed during the High 
School Geography Project to guide teachers and other participants 
through the formulation of learning objectives, evaluation of those 
use of that evaluation to improve teaching, and the 
of the process to the construction of an evaluation 
After completing the process, participants should be 
to apply evaluation data to asseej and improve 
instructional procedures, measure student attitudes toward school and 
particular topics of study, and write essay and objectives questions 
to measure higher level cognitive growth in students. The booklet 
contains an instructor's guide and a participant's manual. The guide 
outlines the five parts of the process, specifying purpose, 
procedures, and materials needed, which include game materials and 
films. The instrpctor's role in obtaining effective learning is then 
presented part by part. The participant's manual provides instruction 
for the various processes and specific content information. The five 
parts are flexible and may be consolidated or divided to accommodate 
available instructional periods. Films mentioned in the document are 
available through the rjatelogs of th<=> Mountain Flains Educational 
Media Council and the Indiana University Audio-visual Center, under 
♦•he title High School Geography: Ne»w Tnniahts. (JH) 
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1 

Rationale for Using Evaluation to Improve Instructio n 



^ Evaluation has often been equated with testing for 

the purpose of grading. The grading process makes dis- 
tinctions in the relative accomplishment of individual 
class members on important course objectives. Grading 
is a necessary and difficult task that is not often done 
well. Evaluation is often regarded solely as this pro- 
cess of assembling and administering tests at the con- 
clusion of an instructional unit in order to have a 
tangible basis for assigning students' marks. The pre- 
occupation with the administrative function of evalua- 
tion, however, has obscured the use of evaluation for 
another and possibly more important purpose. As Jerome 
Bruner has noted, "What is often overlooked is that 
examinations can be allies in the battle to improve 
curricula and instruction."* 

Evaluation procedures can provide feedback about 
the effectiveness of the materials and strategies used 
in teaching. When an instructor determines how his 
group performs on specific objectives before and after 
his teaching, he obtains useful information that can 
help him make his objectives more realistic, or modify 



J.S. Kruno/, The Process of Edu cation. Cambridge: 
Harvard University Press, i960, p. 30. 
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the materials and strategies he has employed. The 
result in both cases should be an improvement in teaching 
and, very likely, an improvement in learning by his 
students . 

The use of evaluation data as feedback for improving 
instruction is likely to have other results, too. When 
grading is the only function of a teacher's evaluation 
program, his testing is likely to be limited to measuring 
the attainment of objectives which can be readily graded. 
Objectives involving the recall of information are the 
easiest for which to develop tests and determine grades. 
Attitudes, values and feelings, which are consistently 
postulated as the most important objectives in educa- 
tion, cannot ethically be graded. Nor can attitudes, 
values and feelings be measured reliably when the student 
realizes that he will be punished or rewarded on the 
basis of his responses. Thus > '-he teacher who evaluates 
only for the purpose of grading probably will not test 
for attitudinal or affective objectives. 

However, the teacher who uses evaluation procedures 
to assess his own teaching can ethically measure affec- 
tive objectives. Group data derived from such attitud- 
inal measures as the semantic differential or Likcrt 
scales arc reliable and valuable feedback. The added 
value of using tests as feedback is that a teacher's 
testing program communicates to the student, better than 
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anything else a teacher does, what is important in his 
course. When affective objectives are important to a 
teacher but his tests ignore them, the student will not 
be convinced of their significance. 

This kit focuses on evaluation procedures that will 
help participants improve their own teaching. In the 
process, participants become acquaint?' 1 with some spe- 
cific techniques that will help them in the more tradi- 
tional use of evaluation. The kit will also provide 
evaluation procedures for some of the more difficult 
objectives to measure, such as higher level cognitive 
and affective objectives. The experience is intended to 
broaden teachers' knowledge about the kinds of learning 
that are measurable as well as to give teachers some 
appreciation for the reason these learnings should be 
measured. 

Overview of the Kit 

In the "Game of Farming," Part I, paiticipants 
become involved in an agricultural investment game in 
which they simulate farmers during two different time 
periods in American history. This part will take 75 
minutes to teach. 

"The Objectives of the Game of Farming," Part II, 
begins with a class discussion in which participants are 
asked to formulate some objectives of the simulation 
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game. They are assisted by viewing a video tape of high 
school students playing the game. After comparing their 
list of objectives with a list devised by the game 
developers, participants discuss the significance of 
some of the important learning outcomes. This part will 
take 4 5 minutes. 

In Part ill, "Evaluating the Objectives," partici- 
pants study an evaluation instrument that measures the 
game's objectives. Participants read and discuss the 
items and scales that compose the instrument. If the 
readings are assigned to be done outside of class as is 
suggested, this part will require 45 minutes. 

In Part IV, "Using Feedback Evaluation," partici- 
pants work and discuss a programmed exercise which 
demonstrates how evaluation data can be utilized to 
improve teaching. This part will take 45 minutes. 

In Part V, "Application," participants construct 
an evaluation instrument for another classroom activi- 
ty. First, they are given the objectives of an activity 
and then, through video tap^, view the activity being 
taught to students. Then participants construct an 
evaluation instrument. This part requires 90 minutes 
to teach. 

The preceding outline and the Overview Chart on 
pages 4 and 5 have been included to tfivc you a summary 
of t he p rocedur cs f tea eh i t iinc and ma t er i a 1 s comp r i s ing 
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this kit. You will note that about five hours of class - 
room time are needed to complete the kit. 
v ^ The five parts of the kit are flexible so that they 

a® may be combined into larger sections or divided into 

4? 

smaller segments depending on the length of your class 

period. For example, if you have one hour class periods, 

your five periods could be organized in this way: 

meeting 1 - introduction and play a part of the 
"Game of Farming" 

meeting 2 - complete the latter part of the"Gamc 
of Farming» M detcrmine the objectives 
of the game and view and discuss the 
video tape 

Assign the class to study the evaluation instrument and 
its accompanying reading between meetings 2 and 3. 

meeting 3 - discuss the evaluation instrument and 
begin work on the programmed exercise 
on evaluation 

meeting 4 - discuss the programmed exercise and 
begin the application 

meeting 5 - complete t^e application 

If you meet for one and one half hour class periods, 
you would need three and one half class periods to com- 
plete the kit. In this case, your time would be organized 
in this way: 

meeting 1 - introduction, play the "Game of Farming," 
and bc*in determining the objectives of 
the game, view the video tape. 

Assign the class to study the evaluation instrument and 
its accompanying reading between meetings 1 and 2. 

meeting 2 - discuss the evaluation instrument, and 
work through and discuss the programmed 
exercise on evaluation 
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meeting 3 - begin the application section 

meeting 4 - complete the application section in the 
first half hour of class 



If your group meets for three hour periods, the full 

time of two meetings will be required to complete the 

kit. Your sessions might be organized in this way: 

meeting 1 - introduction, play the "Game of harming," 
discuss the objectives of the game and 
view the video tape, assign the study 
of the evaluation instrument and its 
accompanying reading to be done in class 

meeting 2 - discuss the evaluation instrument, work 
through and discuss the programmed 
exercise on evaluation and do the appli- 
cation. 



Participant Objectives 



After completing this kit, the participants should 
be more likely to and better able to: 

1. Apply evaluation data to assess and improve 
their instructional procedures. 

2. Measure student attitudes toward school and 
particular topics of study. 

3. Write essay and objective questions to measure 
higher level cognitive growth in students. 



9 



Planning Suggestions and Materials Check List 

The Overview Chart on pages 4 and 5 of this Guide 
lists activities and materials for each part of the kit. 

■ 

For case of reference, some advance planning suggestions 
and the materials required for each part of the kit are 
also described below. 

For Part I of the kit you will need the "Game of 
Farming" materials. These include: 

1 - envelope entitled "Student Materials, First 
Entry 1880-1882" 

1 envelope entitled "Student Materials, Second 
Entry 1919-1921" 

Because the directions for playing the game are com 
plex, you will want to familiarize yourself with them in 
advance of class. 

For Part II you will need: 

1 - Small reel of video tape, "Students Playing 
the Game of Farming." 11 minutes 

30 - Handout sheets, "Some Objectives or Desired 
Learning Outcomes for the Game of Farming." 
These are found in the envelope entitled 
"Separates for the Evaluation Kit." 

You will want to caution participants not to lose 
the handout sheets after you have distributed them for 
they will be needed again in Parts III and IV. 

Remember that between the class meeting in which 
Part II is concluded and the meeting in which Part III i 
begun, you will want participants to do the assignment 



outlined on page 31 of this Guide, and on pages 18 to 
19 of the Participant's Manual. 

For Part II and Part V you will need a video tape 
console and monitor. Also, before teaching these parts 
you should be certain you have the impropriate tape 
ready to use. In this kit you will use portions of two 
different video tapes. Tho tape of students playing the 
■■Game of Farming" is not contained on the reel you will 
use in Part V of this kit. 

For Part III you will need: 

- the "Evaluation Instrument for the Game of 
Farming," on page 20 to 25 of the Partici- 
pant's Manual. 

- the reading, "Constructing and Interpreting 
an Evaluation Instrument," on pages 26 to 41 
of the Participant's Manual. 

30 - handout sheets, "Some Objectives or Desired 
Learning Outcomes for the Game of Farming. 
This sheet was distributed to participants 
in Part II. 

For Part IV you will need: 

30 - copies of the seven-page "Programmed Exercise 
on Evaluation," found in the envelope entitled 
"Separates for the Evaluation Kit.' 

30 - copies of the "Answer Sheet for the Programmed 
Exercise on Evaluation," found in the envelope 
entitled "Separates for the Evaluation Kit. 

30 - handout sheets, "Soirc Objectives or Desired 
Lcarninp Outcomes for the Game of Farming. > 
This sheet was distributed to participants in 
Part II. 

For Part V you will need: 

1 - video tape, Volume SS 81?., "The Classroom As 
It Is," Part III. "Port svi lie." 15 minutes 



n\0 
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10 -"New Orleans East' topographic maps, packaged 
separately. 

? "Educational Objectives of Portsville," be- 

^ ginning on page 44 of the Participant's Manual 

& "Holly Bay Community" sketch map, on page 47 

& of the Participant^ Manual. 
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Your participants will need the topographic maps to 
write test items for the application in Part V. Ask 
them to share the maps since there are not enough for 
the entire class. 



PART I: THE GAME OF FARMING 

Begin Part I of the kit by expJaining to your par- 
ticipants that for approximately 75 minutes they 
.ill be involved in a role-playing simulation exercise, 

jhc Gar.c of Farming." (It is contained in the High 
.Jiool Ge-^raphy Project's unit, Manufacturing and Agri- 

jlturc, rublished by The Macmillan 'Company.) They 
.hould participate in the simulation exercise as adults 
mJ not act as they believe high school students would. 
; -.owever, inform your group that as they go through this 
exercise * they should be conscious of the things students 
-.ight learn from participating in it. When the exercise 
js completed, participants will analyze it to determine 
the learnings that have resulted. Later participants 
will see a video tape of high school students involved 
m the ga-ne. 

The nain purpose of this exercise is to illustrate 
some of -he objectives — cognitive and af f ective- - of the 
simulaticn. Therefore, you should not get involved in 
the details of the game so that you lose sight of the 
purpose in playing the game. While the instructions may 
•ppoar ccr.plcx, the actual playing of the "Game of 
i aiming" naves rapidly once they are understood. 

In the high school classroom version of the simu- 
1 it ion, - iirs of students assume the role of a farmer in 



Western Kansas during three different time periods--1880-82 , 
1919-21, and 1933-35. They are given specified amounts of 
land and money and then decide how to invest their money 
and manage their farm. After they make their yearly 

m 

investment decisions, they learn how accurate these 
decisions were. 

The selection of Western Kansas, a region of variable 
rainfall, demonstrates the farmer's dependence upon 
nature and some of the risks he takes. The time periods 
of the game illustrate the effects of natural conditions 
such as weather, plant and animal disease and insects, 
as well as market conditions. In a high school classroom 
the exercise is planned to take four and one-half to six 
hours . 

However, your participants will play only the first 
two entries or time periods of the three time periods in 
the game. In these entries they will farm during the 
years 1880, 1881, and 1882 and again during 1920 and 
1921. In this abbreviated form, your class should need 
no longer than 60 minutes to play. About 15 minutes should 
be allowed for class discussion. 

Materials 

Participant, information for each of the time periods 
begins on page 5 of the Participant's Manual. 

The following materials arc provided for this exercise: 
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One large envelope entitled "Student Materials, 
First Entry, 1880-1882" that contains: 
1 - envelope labeled "1880" with: 

20 activity sheets 

20 role cards 

24 outcome cards for 1880 
1 - envelope labeled "1881" with: 

20 activity sheets 

24 outcome cards for 1881 
1 - envelope labeled "1882" with: 

20 activity sheets 

24 outcome cards for 1882 
One large envelope entitled "Student Materials, 
Second Entry, 1919-1921" that contains: 
1 - envelope labeled "1920" with: 

20 activity sheets 

24 outcome cards for 1920 
1 - envelope labeled "1921" with: 

20 activity sheets 

24 outcome cards for 1921 

Major Sections of the Exercise 

The two major parts of the game are: 
1. "Playing the Game, 1880-1882." Participants 
assume the roles of pioneers and homestead a farm during 
this period of the settlement of Western Kansas. Their 
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first farming decisions are made on the basis of previous 

« 

farming experiences \' 'ch are described on their role 
cards. In subsequent ars participants modify their 
crop or livestock investment decisions because of their 
success or failure or that of their neighbors. About 
30 to 40 minutes are needed to play this section. A very 
brief class discussion precedes the next section of the 
game . -* - : "-•*•* - 

2. "Playing the Game, 1920-1921." During this 
period farm prices were inflated as a result of World 
War I and farmers prospered and expanded their operations. 
As the European market diminished after the war, the 
slump in prices had a serious effect on the Kansas far- 
mer. The participants will have more information with 
which to work-- information on rainfall, crop yields and 
market prices for the intervening years between the 1880's 
and the 1920 ! s. Now the farms are larger; they have more 
money, different equipment, and a larger choice of crops 
and the option to irrigate. This second time period 
will take approximately 20 minutes. A brief discussion 
also follows. 

Playing the Game, 1880-1882 

To begin the simulation, you should ask the partici- 
pants to read the directions for playing the game on 
pages 5 to l() of the Participant's Manual. These direc- 
tions will also help you manage the operation of the game. 
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After participants have read the directions, you 
should pass out the role cards and instruct the partici- 
pants that they will be working in pairs. Each pair 
^ assumes the role of one farmer to homestead their new 
J" farm during 1880, using past farming experience as out- 

* lined on their role card. After participants complete 

~ ~ thtr _5 r a fil _ a ' rH i 188 2~ time, periods^ „they c~an modify the.ir . . 
crop or livestock investments as a result of their 
previous success or failure. Also, point out the rail- 
road leaflet, on page 7 of the Participant's Manual, 
which gives the kind of information available to pioneers 
in this area in the 1880 's. You should know that this 
railroad leaflet is not entirely accurate. It is more 
descriptive of Eastern Kansas than Western Kansas where 
the farming in the game takes place. 

Next, you will want to distribute the work sheets 
entitled, "1880-]882 Activity Sheet," to each pair of 
participants. Have them fill in the top of the activity 
sheet according to the instructions on page 8 of the 
Participant's Manual. This includes the background from 
their role card, the year (1880), and the "capital," 
their total amount of money ($1500). A sample activity 
sheet is on page 17 of this Guide. You will want to 
refer to this sample as you read the directions for the 
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Have the participants note the crops and livestock 
they have available and the costs per 40-acre units. 
Participants should decide which crops they will grow 
^ and fill in the blank 160-acre farm diagram to the right 
00 of the activity sheet. They should also put the amount 

of money invested in each crop or livestock by that item 
in the "Allocations"column." Point "out that dairy cattle, 
beef cattle and sheep investments are in $100 units with 
a $400 maximum for only one 40-acre unit. The livestock 
may be mixed in $100 units or any 40 acres. Hogs do not 
require special land and may be invested in $100 units up 
to $600 for their farm. Also note the $200 extra expense 
for dairy cattle the first year and the automatic $500 
cost of living expense. Participants may put money in 
' savings with no interest return and rent extra land at 

$100 for 40 acres. 

When participants understand the mechanics of the 
allocations, you should let them work for 5 or 10 minutes 
to decide on their crops and livestock to invest their 
money. The sample activity sheet, on page 17 of this 
Guide, indicates that the German farmer raised 40 acres 
of wheat, 40 acres of barley and 40 acres of rye, each 
costing $100. In addition, he devoted 40 acres to pas- 
ture on which he grazed $200 worth of dairy cattle and 
raised $300 worth of hogs which required no special land. 
Five hundred dollars was allocated for cost of* living 
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and he had to pay $200 for his initial investment in 
his dairy herd. His total expenditure was $1500. Partici- 
pants should go through a similar decision-making process 
in completing thjeir activity sheets. 

Figuring Outcomes 

When the participants have made allocations for 
their farm, you should refer them to the instructions 
on pages 9-10 of the Participant's Manual so they can caltu 
late how successful the first year has been. 

Have each pair of participants draw one " 0ut ~ 
come Card -1880" from the cards provided. They should 
read the introductory paragraph on the card which gives 
a description of the climatic conditions of the area 
and how they affected the farmer's crops. Information 
regarding plant or animal disease or market conditions 
is included. Now tell the participants to note the 
"multipliers" they received for each of the crops and 
livestock they worked with. A multiplier should be 
entered in thc"Mul tiplier"column by the appropriate crop. 
Then tell the participants to multiply the amour' of 
money allocated for that crop with the multiplier to 
determine the outcome for that crop. 

Participants will recognize that multipliers differ 
from card to card. The differences are part of the 
design of the game. The outcomes, based on actual records 



9 

ERIC 



nl« 



20 



* 

from a county in Western Kansas, reflect both the varia- 
tions in natural conditions such as local weather, plant 
and an'mal diseases, and insect damages, as well as 
variations in market prices for particular commodities. 

The sample activity sheet shows that the German 
farmer drew a card that listed a multiplier for wheat of 
^•i--ba.rlcy,-2;. .rye.,. 2;...dairy cattle, S; and hogs, .1. He 
entered these numbers in the"Multiplier"column next to 
the appropriate product. By multiplying the amount of 
money allocated to the various crops by the respective 
multipliers, the farmer figured his income for each 40 
acres. On the sample sheet, the German farmer received 
a $300 return for the $100 invested in wheat; $200 for 
the $100 invested in barley, $200 for the $100 invested 
in rye; $600 for the $200 invested in pasture cattle; 
and he just broke even on his hogs. Of course, he gets 
no return for his $500 for cost of living and he receives 
no return on his savings. His total income, therefore, 
is $1600. ilis net income is $100. He now has $1600 to 
reinvest in his farm operation in 1881. 

At the end of the 1880 round, you should allow the 
* players to talk with their neighbors to find out how 
they fared and to determine why some farmers were more 
successful than others. They should try to obtain infor- 
mation that will help them make more reasonable alloca- 
tions in the second year. 
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The next two rounds of allocation, 1881 a,nd 1882 , 
should take less class time since participants are 
familiar with the roles and with the allocation sheets. 
The procedures for play are the same. 

You will want to pass out a new activity sheet for 
the year 1881. Participants should enter their role at 
the top of the activity sheet and the amount of money 
they' ended with in 1880 '. They" then * make the i r a 1 1 o c a - 
tions for the yea." 1881, draw an outcome card for that 
year, and calculate the return on their investment. 

Again, at the end of 1881 the players should be 
allowed to share information about the results of their 
farm operation and then immediately make their alloca- 
tions for the year 1882. 

Discussion of the 1880-1882 Period 

When participants have figured the results of their 
^arming operation for the year 1882, you will want to 
take time for a short discussion of their experiences. 

The discussion question raised at this time will be 
useful background for participants when they analyze the 
objectives of the game in a later part of this kit. You 
may want to post on the blackboard some of the partici- 
pants' responses to this question. To begin the discus- 
sion ask: 



"From your experience with farming in the 1880* s, 
what kinds of f-.ctors influence a farmer's success or 
failure,?" 

Answers probably will be varied. The first three 
years, however, have been designed to illustrate primar- 
ily physical factors that influence farmers' management 
decisions. During 1880. for instance rainfall was about 
average- £©r the county; that is, about 18 inches of 
precipitation. This is adequate rainfall if it falls 
at the right time for a wide variety of crops. Like 
many marginally arid areas, however, great variations in 
annual rainfall may be expected. In 1881, for example, 
Settler County received 33 inches of precipitation, but 
in 1882 only 7 inches. As a result, those farmers who 
raised crops requiring humid conditions, such as corn, 
sustained substantial losses in 1882. Those who under- 
took a less intensive farm operation, such as wheat or 
grazing, fared better. Also, in marginally arid areas 
precipitation seems to fall in violent storms. So, even 
in years with adequate rainfall, yields may be reduced 
because of wind and hail damage. 

Diseases such as rust and blight, or pests such as 
grasshoppers or corn borers, also diminish yields and 
affect profitability of the farm operation. 

Participants might also point out that their roles 
wiRht have influenced their success or failure during 
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1 the first year of farming. That is, if they came from 

an area that supported humid crops they may not have 
been successful if they tried humid crops in the dry- 

* marginal area.* All of these factors are taken into 

account in the outcome cards so that in the play of the 
game, some or all of the players experienced some of 
these problems. Now you are ready to begin the 1920- 
1921 period. 

Playing the Game, 1920-1921 

You should give the pairs of participants a new 
activity sheet entitled "1919-1921 Activity Sheet" and 
refer them to the directions on pages 10-12 of the Partici- 
pant's Maival. (Tell the participants that the year 
1919 is omitted.) 

The participants enter 1920 with a 320-acre farm, 
J $8500 capital, living expenses of $2000 per year, and a 

1 $1000 payment on mortgage and machinery. Bach farmer 

\ has a hired laborer earning a salary of $600 per year. 

Jiy 1920 the farmers have much more knowledge about 
Western Kansas than the original homesteaders had. Much 
of this information is summarized on graphs on pages 13 
to 16 in the Participant's Manual. The graphs explain 
how crop yields relate to rainfall, which crops do better 
in dry years, and the relationship between crop yield 
and crop price. Be sure to remind your group of the 
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availability of this data because an improved ability to 
read and interpret graphs is an objective of the game. 

The procedures are the same as for the 1880' s. 
Allocations are made in 80-acre units only, and you may 
have to point out that the diagram of ^ farm on the 
activity sheet shows the 80. acres outlined in solid 
black with a dotted line dividing it into two 40-acre 
parts. 

Players have a choice of irrigating up to 80 acres 
of their land, and planting sugar beets, alfalfa and 
corn. There is an expense of $200 for the first year 
to the farmer who irrigated land. Alfalfa is the only 
crop that may be chosen for either irrigated or unirri- 
gated land. All other crops are either exclusively ir- 
rigated crops, e.g., sugar beets, or corn, or unir- 
rigated crops, e.g., wheat, barley or sorghum. 

The livestock choices are the same as in the 1880's. 
As in the earlier period, there is an additional fee of 
$200 for the first year's investment for dairy cattle. 
Players may invest up to $2400 in beef cattle, dairy 
cattle or sheep. They may invest up to $600 in cattle 
or sheep on any 80 acres of land. There is no limit to 
the amount of investment in hogs. Land may be rented for 

$800 per 80 acres. 

You will again let participants draw outcome cards 
for the years 1920 and 1921 so they can calculate the 
success or failure of their farming. 



25 

At the end of the 1920 time period, you should let 
the participants discuss their outcomes with their 
neighbors in order to share information and learn from 
one another. 

Discussion of the 1920-1921 Period 

When the participants have completed the 1920-1921 
years, you will want to conduct again a short discussion 
to summarize some of the learnings that have taken place. 
Answers to discussion questions also will be useful to 
participants when they analyze the objectives of the 
game in Part II of this kit. 

Hcgin by asking a question such as: 

"What would you add to the list of factors that 
influence a farmer's success or failure?" 

Responses to this question will vary. Participants 
should point out that new factors such as economic 
markets, political conditions, and new technology had to 
be taken into account during this time period. They 
should also point out that they had more knowledge about 
the area from the charts and graphs about climatic and 
economic conditions than they had available during the 
1880's, 

Another question you might wish to pose is, "Can 
you make any kinds of comparisons between farming and 
another type of economic activity, such as manufacturing, 
from having been involved in this game?" 
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The answers you get to this question will depend 
<f upon the sophistication of the group. Similarities and 
# differences such as the following examples probably will 

oaf be mentioned. 

1. Both farming and manufacturing require large 

capital investments. 

2. Farming is a larger space user than manufac- 
turing . 

3. The farmer has less control over the quantity 
and quality of what he produces than the manufac- 
turer. 

4. The farmer has less control over the price he 
receives for his product than the manufacturer. 

5. Farming is more of a seasonal activity than 
most forms of manufacturing. 

Finally, you may want to determine if participants 
understand the principle of scarcity and how it functions 
in an agricultural context. Ask your class: 

"From your experience as farmers, do you sec any 
relationship between the quantity of an agricultural 
crop proluced and the price that is paid for that product?" 

The answer should be apparent but if it is not, the 
general relationship can be illustrated with the graphed 
data in the Participant's Manual. You could pick a crop 
on the graphs and show that, in most cases, as the quan- 
tity produced increases, the price of the crop drops. As 
the quantity of the crop decreases, the price goes up. 
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PART II: THE OBJECTIVES OF THE 
GAME OF FARMING 

This part of the kit focuses on the variety of 
objectives of the "Game of Farming." First, partici- 
pants will determine the objectives. Then, they view an 
eleven-minute video tape of high school students partici- 
pating in the game. The video tape helps participants 
determine what students are learning. Then they discuss 
the significance of some of the less easily recognized 
objectives of the game. Finally, they compare a list of 
objectives devised for the game by its developers with 
the list the participants have made. 

The materials needed for this part include: 

1 - The 11-minute video tape of the "Game of Farming," 
on the smaller of the two reels of video tape 
provided by the ACM Video Tape Project. 

30 - Handout sheet, "Some Objectives or Desired Learn- 
ing Outcomes of the Game of Farming." 

You will have arranged previously for a video tape 
console and monitor to be used during this part. You also 
should preview the video tape of the "Game of Farming." 
Scanning the handout sheet may be helpful, too. 

Initiate this part of the kit by asking the partici- 
pants : 

"What do you think students will learn Trom partici- 
pating in this simulation game?" 



As suggestions of objectives for the "Game of 
Farming" are made, list them on the chalkboard. 

When the group has given you all the. game's objec- 
tives that they can determine, introduce the video tape. 
Tell the participants that the eleven -minute video tape 
shows the game being used in an American history class 
with a group of eleventh graders. As they watch the tape, 
participants should look for additional game objectives 
that they have not listed previously. If participants 
have not already listed objectives in the affective 
domain, the video tape should make them aware of = omc of 
these . 

When the video tape has been played, ask the class: 
"Having viewed the tape, are there any objectives 
that you would add to your list?" 

Near the end of the video tape sequence, the teacher 
asks his class if they can tell him what they have learned 
from having played the game. One of the boys in the 
class indicates a newly acquired feeling about farmers. 
Reacting to being wiped out financially in the depres- 
sion, he comments that after farming his land for fifty 
years he would not leave it under any conditions. The 
student demonstrates a real empathy for the farmers who 
were forced to leave their land. Participants arc likely 
to remember this sequence on the tape and it could serve 
as tne stimulus for a discussion of attitudinal objectives. 
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Questions similar to the following might be useful to 
guide the discussion. 

"Are feelings, such as those expressed by the boy 
in the video tape, as important objectives as the other 
objectives you have listed?" 

Among educational theorists, attitudes, feelings 
and values are constantly postulated as the most 
important outcomes of education. Courses are often 
justified on the ground that they will have an impact 
on students' attitudes long after the specific knowledge 
taught in the course has been forgotten. Continue this 
line of questioning by asking: 

"How would you describe the way this group of stu- 
dents feel about the experience they were involved in?" 

The majority of the class seemed to be interested 
in what they were uoing and enjoying themselves. Now ask: 

"Are student feelings or attitudes toward the sub- 
jects they study important objectives?" 

As one educator recently noted, if students learn a 
great deal but dislike the experiences they are having, 
this is likely to be a deterrent to future learning. At 
a time when the continuance of learning is so necessary, 
teachers must be conscious of the attitude that is 
developing toward any subject being studied. 

When this discussion is completed, hand out "Some 
Objectives or Desired Learning Outcomes of the Game of 



Farming." Explain that the list was compiled by the 
developers of the game. Ask your participants to com- 
pare their objectives with those of the game developer 
and to note any differences. 

The participants may have suggested objectives not 
on the game developer's list. Reassure them that the 
game can teach a broad range of objectives. Only some 
of the key objectives have been included by the game 
developer. If the participants have been involved in 
other HSGP teacher education kits, they may consider 
decision-making as an objective. It is a legitimate one 
if the teacher emphasizes decision-making in the play of 
the game. However, decision-making has been omitted from 
the game developer's list because it would unduly lengthen 
this discussion. 

On the other hand, participants probably will not 
have thought of some of the objectives on the game devel- 
oper's list. The class may want to discuss where in the 
game some of these learnings would take place. 

When the participants are satisfied that they under- 
stand the objectives of the "Game of Farming," make the 
transition into Part 111 of the kit by asking: 

"Would you have any difficulty evaluating student 
attainment of the objectives of the Game of Farming?" 

Participants will probably agree tha l evaluating 
su-h a wide variety of objectives presents a complex 
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problem. In Part III participants vill look at an evalu- 
ation instrument designed to measure the objec-ives of 
this simulation game. Tell the participants to keep the 
handout sheet, "Some Objectives or Desired Learning Out- 
comes of the Game of Farming," as it will be used in the 
next parts of the kit. 
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PART III: EVALUATING THE OBJECTIVES 

In this part of the kit, participants are assigned 
to study an evaluation instrument for the "Game of 
Farming." The" assignment has two parts, both of which 
can be completed out of class. First, participants are 
asked to match test items with the specific objectives of 
the game. Then they study. a reading which gives some 
specific suggestions for constructing the various kinds 
of items and scales found in the instrument. When the 
assignment has been completed, a class discussion high- 
lights the important ideas. 

To complete this part of the kit in the 45 minutes 
suggested, the assignment found on pages 18 to 19 of the 
participant's Manual will have to be completed out of 
class. Review the assignment instructions so that you 
are familiar with what the participants will be doing. 
You will also want to preview the evaluation instrument 
and the reading on the construction and interpretation 
of the instrument. 

When the group meets after having completed match- 
ing test items with objectives and the reading, you will 
want to conduct a class discussion about the assignment. 
As the discussion is carried on, the participants should 
have in front of them the evaluation instrument, the ob- 
jectives of the game, and the reading on construction 
and interpreting an evaluation instrument. 
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You might begin the discussion by asking:. 

"You have tried to match the items on the instru- 
ment with the objectives of the game. Does the instru- 
ment measure the objectives of the game?" 

The intention is to measure every objective by at 
least one test item. For your information, the objec- 
tives and items arc matched below. You might post this 
chart so that the participants can compare it to the 
chart they were asked, to complete in the assignment. 

Objective Number Test Item Number 

1 2,3,4,7 

2 1,6 

3 8 

4 5 

5 9 

6 11,13,16 

7 15,17,18,19,20 

Items 10, 12 and 14 do not relate to objectives of the 
game but provide reference data for use with items 11 
and 13. This practice was explained on page. 33 in the 
reading in the Participant's Manual. 

To determine if there are ideas in . the assignment 
which were not understood, you could ask questions 
similar to these examples. 

"In doing this assignment you became familiar with 
a variety of testing techniques. Some of these were 
known to you before and some you were introduced to for 
the first time. Could you now construct these items and 
scales yourself?" 
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Because participants will have to construct an instru- 
ment like this in Part V of the kit, you should tr> to 
clarify any questions that they may raise. At this time, 
forestall any discussion of the uses of these instru- 
ments. This question should be raised later as a tran- 
sition to Part IV. 

When you have answered their questions regarding 
the . construction of the items and scales, continue the 
discussion by focussing attention on Part 1 of the instru- 
ment. Remind the class that earlier in this kit, they 
discussed the relative importance of certain kinds of 
objectives in comparison with other objectives. Test 
items can be ordered in terms of relative importance, 
too. To bring this idea to the class's attention, ask: 

"Are the questions included in Part I of this 
instrument measuring learnings of equal significance, 
or do some questions seem to be measuring n.ore important 
learnings than other questions?" 

If participants will look carefully at items 1 
thiough 9, they will see that some of the items are 
measuring relatively trivial things. If this is not 
immediately apparent, however, have participants compare 
items 5 and b . You might ask them: 

"Which of these questions (5 or 6) seems to be 
testing a more important learning? Why?" 
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Participants probably will see that question 6 
.neasures a relatively insignificant piece of information 
which is not likely to be of much use to the student 
who knows the answer. To answer question 5, however, 
the student will have to understand at least some of 
the principles involved in produce marketing. To some 
extent those principles can be generalised and, there- 
fore, useful to the student in other contexts. 

It may be o£ value to continue this line of ques- 
tioning by asking the group: 

"If your tests emphasized questions like number 6, 
would this have any influence on the way your students 

would study?" 

If ; teacher's examinations require the memoriza- 
tion of specific pieces of knowledge. the teacher should 
not bo surprised to find his students preoccupied with 
the extraction of minutiae from their textbook and class 
notes . 

At this point, you should shift the attention of 
your class from Part I of the evaluation instrument to 
the attitude measurement scales that are represented in 
Parts II, HI, ^d IV. These scales are used for dif- 
ferent purposes than the items in Part I. and they are 
subject to somewhat different interpretation procedures 
You should be sure these differences are clarified for 
the participants. 
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This part of the discussion should provide a tran- 
sition into Part IV of the kit which deals with the use 
of feedback evaluation data. To conclude the class dis- 

cussion ask: 

"Will all the parts of this instrument provide 
information that is usable to grade students?" 

You should be absolutely certain that participants 
understand that the data derived from Parts II, III, and 
IV of the instrument cannot be used for grading purposes. 
The ethical problems involved in grading attitudes or 
appreciations should be obvious. There is also the 
problem of destroying the reliability of these instru- 
ments by rewarding or punishing students for their 
responses on them. When grades are at stake, items in 
Parts II, III, and IV are extremely vulnerable to a 
student's faking the attitude his teacher seems to want. 
When this idea seems to be understood, continue the 
questioning by asking: 

"Since the items in Parts II, III, and IV cannot 
be used for grading, they must be included for some 
other evaluation purpose. Did your reading suggest any 
other uses for evaluation data besides grading?" 

The reading did allude to the use of these items 
and scales in assessing the effectiveness of the mate- 
rials and teaching strategics being used in the class- 
room. 
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Indicate to participants that in Part IV they will 
study the use of data from such scales for "feedback 
evaluation." That is, the results of the evaluation can 
indicate ways, to modify teaching procedures. 
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PART IV: USING FEEDBACK EVALUATION 

In this part oT the kit, participants arc involved 
in a programmed exercise which demonstrates the use of 
evaluation data as feedback to improve teaching. The 
exercise, "A Programmed Exercise on Evaluation," will 
';takc.:.participsnts .about. 20 : minutes_. to work ^and- could.be 
done out of class. The discussion which follows the 
program will take another 25 minutes. 

The materials needed for this part of the kit in- 
clude: "A Programmed Exercise on Evaluation," the 
"Answer Sheet for the Programmed Exercise on Evaluation/ 
the handout sheet, "Some Objectives or Desired Learning 
Outcomes of the Game of Farming," which was distributed 
in Part II, and the "Evaluation Instrument for the Game 
of Farming." All of these materials, except the evalua- 
tion instrument, arc packaged separately in an envelope 
labeled "Separates for Evaluation Kit." The evaluation 
instrument is found on pages 20 to 25 of the Partici- 
pant T s Manual . 

You will want to read through the program before 
you distribute it to participants. Directions for its 
use are attached. When you distribute the program and 
the answer sheet which accompanies it, be sure your 
class members have in front of them the handout sheet, 
"Objectives or Desired Learning Outcomes for the Game 
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of Farming," as well as the evaluation instrument. These 
materials are referred to in questions incorporated in 
the program. . . 

If your group is unfamiliar with the way a linear 
program operates, explain that even though they are 
working with an answer sheet, the exercise is not a test. 
The answer sheets will not be looked at by anyone other 
than themselves. The answer sheet functions to make 
participants concentrate for a few seconds on the ques- 
tions before the answers are provided. 

When the class has completed the program, you will 
want to discuss its contents to ensure that the major 
ideas have been understood. The following questions 
can guide this discussion. 

"According to the programmed exercise, what are 
some specific ways that evaluation data can be used to 
improve teaching?" 

The exercise mentioned the utilization of pretest 
results to determine if instructional objectives are 
realistic. Pretest results also can show if objectives 
already have been satisfactorily accomplished. To deter- 
mine which objectives have not been accomplished, you 
can compare pretest to posttcst gain scores on test items 
related to the specific objectives. This technique, if 
applied to both knowledge and attitudinal objectives, 
will indicate where materials or particular teaching 
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procedures are not doing the job intended. When these 
uses of evaluation data have been mentioned, continue 
the discussion by asking: 

"Do you see any difficulty in utilizing the eval- 
uation procedures suggested in this program in your 
classrooms?" 

Several types of problems, jnay. ".occur to "pa'rticipan • c . 
One problem is likely to be the time factor involved in 
devising an evaluation instrument as complete as the one 
suggested. Considerable time is needed to isolate specif- 
ic objectives and write test items that will provide 
precise data about the accomplishment or lack of accom- 
plishment of these objectives. The need for time will 
be especially apparent if the broad range of objectives 
discussed in the program are evaluated. 

Finally, you might wish to ask: 

"Do you think the suggested evaluation procedures 
would improve a teacher's teaching? If so, why? If 
not., why not?" 

When you feci you have adequately discussed the 
ideas in the programmed exercise, you will be ready to 
go on to Part V of the kit. 
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PART V'. APPLICATION 

During the final two hours of this kit, partici- 
pants can apply some of the evaluation ideas and tech- 
niques that they have learned about in previous parts 
of the kit. in this application, the participants are 
given the objectives for another classroom exercise and 
view a video tape of students participating in it to see 
how the objectives are taught. Then participants devise 
and critique evaluation items to measure the attain- 
ment of the objectives. 

There is considerable flexibility in the way the 
application can be handled. If you have an experienced 
group of teachers who are already knowledgeable about 
test item writing, you may want to make the item- writing 
part of this application an out-of-class assignment. 
If, on the other hand, your group is less experienced, 
y ou will want to give much more careful directions and 
guidance and will probably want more of the work done in 
class. If the whole application assignment is done out- 
side of class, only 45 minutes of class time will be 
needed for the opening and concluding discussions. If 
greater guidance is given, the application might require 
about 90 minutes of class time and another hour out of 
class . 
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Materials 

^ 1 Video tape, Volume SS 812, "The Classroom As It 

^ Is," Part III. "Portsville." 15 minutes 

"Educational Objectives for Portsville," pages 44-46 
of Participant's Manual 
~ 10 "New Orleans East" topographic maps 

"Holly Bay Community" sketch map, page 47 of the 

Participant's Manual 
Materials necessary but not provided include video 

tape console and monitor. 

Regardless of whether you intend the application 
to be done in or out of class, begin by directing the 
participants' attention to Part V, beginning on page 43 
of their Manual, and to the section entitled, "Educa- 
tional Objectives for Portsville." (The"Portsville"exer- 
cise will already be familiar to the participants if they 
worked the kit, Using Simul ati on to Involve Students ., in 
which they viewed a video tape of students involved in 

"Portsville.") 

Now tell the participants they are to devise items 
to measure these objectives. Since the "Portsville" 
objectives include attitudes as well as cognitive learn- 
ings, the application should give the participants some 
experience developing measurement instrument? like those 
they examined for the "Game of Farming." 
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The class will need to review the video tape of 
"Portsville" in order to remember the context in which 
the objectives are conveyed. You can show the video 
tape as soon as the participants have read the objec- 
tives for "Portsville" and understand that they are to 
devise test items to measure these objectives. About 
15 minutes are needed to show the tape. 

After participants have viewed the tape, you will 
want to outline the number and kinds of items and scales 
you want them to construct. This assignment is flexible, 
fitting your time constraints. If you ask the group to 
write one or more test questions for each objective, 
the assignment will take a considerable amount of time. 
It is suggested, instead, that participants concentrate 
their attention on two or three objectives and attempt 
to write one or two types of questions for each objective 

Initiate your explanation of the assignment by dis- 
cussing the extra materials that are provided for use in 
writing certain items. The "Holly Bay Community" sketch 
map on page 47 of the Participant's Manual, and the 
"Now Orleans liast" topographic map arc provided as infor- 
mation sources to be utilized in writing questions for 
the knowledge and intellectual skill objectives cf "Ports 
vilic." The practice of structuring questions in "a 
fresh setting" was introduced to participants in the 
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reading "Constructing and Interpreting an Evaluation 
Instrument." If students have been taught the prin- 
ciples governing land use relationships through the use 
of a particular map, you may only be testing ability to 
memorize when you use the same map in an examination. 
When you ask students to demonstrate their understanding 
of these principles with another map, the transfer 
raises the level of understanding necessary to answer 
the item correctly. 

Some suggestions for the assignment you might make 
and examples of items which would satisfy each require- 
ment are given below. Participants probably should, at 
least, have the opportunity to devise: 1) a semantic 
differential, 2) a few Likert Scale items, 3) a few 
objective items (particularly multiple choice), 4) an 
essay question. If participants seem unable to write 
items to satisfy a particular part of the assignment, 
you may want to help them by giving them a specific ex- 
ample from the following. 

I. Utilizing the "New Orleans East" topographic 
map, write a multiple choice question that measures 
objective seven. 

The fol Lowing is an item which would satisfy this 
requirement; 

"Which of the following would not represent an 
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example of man's aoility to adapt his environment?" 

A. The levees along the Mississippi River 

B. The inner harbor navigation canal (in 
the northeast corner of the map) 

C. The pumping station (29°54' N and 90°01' W) 
*D. Bayou Fatima (29°52» N and 90°03» W) 

2. Write an essay-type question for objective 
seven that also utilizes the "New Orleans East" map. 
Include a model answer as was suggested in the reading, 
"Constructing and Interpreting an Evaluation Instrument," 
page 30 of the Participant's Manual. 

The following is an item which would satisfy this 
requirement : 

Find three examples on the map that demonstrate 
how man can modify his physical environment or adapt to 
it. With each example, indicate the probable reason 
the modification or adaptation was made. (250 words, 
30 points, 25 minutes time,) 

An outline of a model answer for this essay might 
be similar to the following: 

A. The levees along the Mississippi River 
represent a human modification of the physical 
environment to keep the meandering river in a 
channel, 

H. The many pumping stations shown on the map 
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represent adaptations to keep dry the areas of 
the city that are below sea level. 
\. C. The many canals shown on the map represent 
adaptations that facilitate the transport of 
goods between the Mississippi River and Lake 
Pontchar train and other bodies of water, 

3. Using the "Holly Bay Community" sketch map, 
construct an objective item to measure objective six. 

The following is an item which would satisfy this 
requirement : 

"If a person walked at the rate of a mile every 
twenty minutes, how long would it take him to walk by 
the nearest route from his home at point 3 on the map 
to the CBD of Holly Bay?" 
A. one hour 
*B, one hour and forty-five minutes 

C. two and a half hours 

D. three hours 

4. Without utilizing any map source in struc- 
turing the question, write a multiple choice item that 
measures the accomplishment of objective five. 

The following is an item which would satisfy this 
requirement: 

"Which of the following is most likely to lead to 
the largest increase of population for a city?" 
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A. Establishment of recreation centers lor 
teenagers 

B « Introduction of a chain of shopping 
centers 

*C. Construction of a steel plant 
D. Improvement of the central business 
district 

5. Utilizing the semantic differential technique, 
construct an instrument to measure objective two dealing 
with the students' attitude toward cities. 

In fulfilling this requirement, th^ participants 
might list the following pairs of adjectives among those 
included in the instrument: rational- irrational , 
sensical-nonsensical , meaningful-nonmeaningful , inter- 
esting-boring, comprehensible- incomprehensible. 

6. Utilizing the "Holly Bay Community" sketch map 
on page 47 of the Participant's Manual, write two mul- 
tiple choice questions that measure objective three. 
The two items should not deal with the same land uses. 

The following arc examples whicli would satisfy this 
rcqui rcmcnt : 
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Questions 1 and 2 refer to the above map. 



In which area of this community is there most likely 
to be low income housing? 

A. 4 
*B. 7 

C. 9 

D. 3 

In which of the following areas is there nost likely 
to be high income residential land use? 

*A. 3 

B. 5 

C. 6 

D. 7 
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7, Write a series of Likert Scale items that 
measure objective two. 

The following is an item which would satisfy this 

requirement : 

Cities exhibit predictable patterns of organization. 

(SA A UN DA SDA) 

When the participants have completed writing the 
items that you have, assigned, you will want to con - ~t 
a discussion to bring some of the interesting items to 
the attention of the whole group. This could be done 
in any of several ways. You might select a few items 
from the participants' papers and write them on the 
chalk board for discussion. Or, you might ask for vol- 
unteers to share items they have written by putting 
them on the chalk board. When you have some examples 
in front of the class, ask the group to critique the 
items in terms of the following criteria: 

1. Docs the item measure the objective it purports 
to measure? 

2. Is the item sound in terms of grammar and 
syntax? 

3. Docs the item violate any of the directives 
provided in the reading, "Constructing and Interpreting 
an Evaluation Instrument?" 

If you have time, participants may find value in a 
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short discussion to review the idea of using evaluation 
data as feedback to improve instruction. Questions 
similar to the following would be helpful. 

1. "If you administered an item related to objec- 
tive six of "Portsville" and obtained no gain for your 
group from pretest to posttest, would you know where in 
the exercise your instruction had been inadequate?" 

2. "Would having this information enable you to 
teach this objective more adequately the next time you 
taught "Portsville?" 

If the video tape of "Portsville" is still fresh 
in the participants' minds, they will remember how the 
teacher introduces the "Portsville" map and discusses 
its scale. Participants should have ideas about ways 
the explanation could be improved if it was inadequately 
presented the first time. 
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General Information 

With the publication by The Macmillan Company of 
the High School Geography Project's year-long course 
Geography in an Urban Age , the Project has turned its 
attention to the development of teacher education mater- 
ials. The kits of HSGP teacher education materials re- 
flect the Project's emphasis on teaching strategies that 
encourage student inquiry and involvement. Like many of 
the HSGP units, the teaching procedures demonstrated in 
the teacher education kits are usable in other social 
studies courses, as well as geography. 

Three HSGP teacher education kits have been pre- 
pared for use both by college students in social studies 
methods classes and by in-service teachers in workshops. 
Each kit is five hours in length and can be used inde- 
pendently or together with the other kits. Some of the 
exercises in each of the kits have been adapted from the 
HSGP course. While these exercises were originally in- 
tended for use with high school students, they have been 
tried out successfully with numerous adult groups also. 
All materials needed, such as aerial photos, maps and 
diagrams arc provided in the kits. Video tapes are also 
available on loan. The three HSGP teacher education 
kits developed for 1970 tryout are: 
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Using Simulation to Involve Students 

Using Media to Stimulate Inquiry 

Using Evaluation to Improve Instruction 

Each kit is based on the assumption that the best 
way to learn the advantages and disadvantages of any teach- 
ing strategy is to experience it. Consequently, the kits 
provide many opportunities for participation in the use 
of a variety of educational media, in role-playing simu- 
lations, and in evaluation exercises. Such experiences 
are subsequently analyzed in terms of their educational 
objectives and their usefulness in a variety of social 
studies classrooms. At the conclusion of each kit, par- 
ticipants are given an opportunity to apply what they 
have learned. 

The HSGP teacher education materials have been 
developed with definite participant objectives in mind. 
Participants completing the HSGP teacher education ma- 
terials should be both better able to and more likely 
to : 

1. Use open-ended strategies, group work, and a 
variety of educational media in their teaching 

2. Analyze inquiry end simulation exercises in 
terms of certain characteristics 

3. Devise inquiry and simulation activities 

4. Evaluate cognitive and affecti- objectives 

5. Discuss the advantages and dis;u' va'ntages of the 
open-ended strategics which the HSG^ teacher educa- 
tion materials demonstrate 
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Introduction to Using Evaluation to Improve Instruction 

To most of us in the teaching profession the term 
"evaluation" recalls the onerous job of administering 
and grading tests. Grading is sometimes a distraction 
that saps time and energy while keeping us from pur- 
suits that might be beneficial to our teaching. In this 
kit you will be introduced to another use of evaluation. 
As Jerome Bruner points out, evaluation can be an ally 
in the battle to improve curricula and instruction. 

The content of the kit is divided into five parts. 
In these parts you will participate in a simulation game 
and then analyze and evaluate it. A brief description 
of each part of the kit follows. 

Part I, Playing the Game of Farming. Working in 
pairs, you will simulate a farmer in Western Kansas dur- 
ing two different time periods of American history. In 
this role you will invest your land and money and then 
determine how you would have fared in each year farmed. 

Part II, The Objectives of the Game of Farming. In 
a class discussion you analyze the game to determine what 
it teaches. You arc aided by viewing a video tape of 
the game being played by high school students. 

Part III, Evaluating the Objectives. You examine an in- 
strument that was designed by the developers of the game 
to measure its objectives. You will learn how to develop 
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and interpret items and scales like those on the sample 
instrument. 

Part IV, Using Feedback Evaluation. You work 
through a programmed exercise which demonstrates how 
evaluation data can be utilized to improve the instruc- 
tional process. 

Part V, Application. You have the opportunity 
to devise an evaluation instrument for another learning 
activity. 
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PART I: THE GAME OF FARMING 

For the next hour and a half you will participate 
in a simulation game that was designed for high school 
students. You should participate in the game as an 
adult, and try to consider the things high school stu- 
dents would learn from it. When you have completed 
the play of the game, your class will analyze it to 
determine what it teaches. ("The Game of Farming" is 
an activity contained in the High School Geography Pro- 
ject's Manufacturing and Agriculture unit published 
by The Macmillan Company.) 

To play the game, you will assume the role of a 
farmer moving to Kansas in 1880. Study the descrip- 
tion of the farmer's past experiences on the role card 
you receive to prepare to farm as you think he would 
have • 

You have selected a 160-acre farm site to homestead. 
You have brought all the necessary farm equipment west 
with you. Since you are opening new land to agriculture, 
there are no farmers to talk to, no farms to visit to 
give you ideas about the crops and methods that suc- 
ceed in this area. You will, therefore, have to rely on 
your past experience as a guide. 

While traveling west you picked up c iailroad leaf- 
let tell inn about Western Kansas. You will find it on 
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page 7 of this Manual. This is all you know about the 
area in which you have settled. 

The first step is to plan how you will allocate 
your money. In addition to the farm equipment, you 
have $1500 to start your farm but you must budget 
$500 for the cost of living during the first year. 
The cost of living must be set aside each year. You 
can spend money in $100 units only. The^ smallest unit 
of land you can work is 40 acres. 

The crops you can plant include wheat, oats, bar- 
ley, rye, and corn. It costs $100 to raise 40 acres 
of wheat, oats, barley or rye, but it costs $200 to 
raise 40 acres of corn. 

You may invest in beef cattle, dairy cattle, sheep, 
and hogs. If you decide to start a dairy herd, an addi- 
tional expense of $200 is required only during the first 
year. On any 40-acre tract you may graze from $100 
to $400 worth of sheep, beef cattle, or dairy cattle. 
You may mix your investment in sheep, beef cattle, or 
dairy cattle on a 40-acre tract, but only in terms of 
$100 units, such as $100 in sheep and $300 in beef cat- 
tle. Hogs do not require special land but .^ay forage 
on crop or pasture land. You may invest from $100 to 
$600 in hogs. 

Additional land may be rented fcr $100 per 40-acre 
unit. If you do not choose to farm all your land, you 
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These Lands are subject to Entry under the Pre-emp- 
tion. Homestead and Timber Acts, and now is the time 
for those who desire to make homes for themselves and 
Their children to secure them. 

The rapid extension of the CENTRAL BRANCH places 
lands within easy reach of all. and durine the present year 
there will be such a rush to this region that those who 
wish to obtain desirable Lands FREE, should so at once and 
secure them. In order that all may be informed of the 
requirements of the laws governing the securing of Gov- 
ernment Land, we have prepared and published in this 
sheet the COMPLETE text of each of the Acts under which 
they are to be acquired, in convenient form for reference. 

The advantages gained by seeking homes on and 
beyond the line of the CENTRAL BRANCH are many, and we 
enumerate a few of them. 

1ST No Land Grant Railroad runs into Inis secnon, hence, rreempiors ana 
Homesteader* are entitled to acquire full quarter sections instead of 
eighty acre tract*. 

2ND— The Central Branch ia rapidly extending its road Ihrough the country com- 
priaed o( theso Lands, and setllers will have the advantage ol being near 
to markets, schools, churches and all other necessities ol civilized people, long 
before they have acquired the title to their Land, and before it is taxable. 
Nc taxe* until title is complete. 

3RD —The country la noted for healthfulness, being composed of beautiful rolling 
prairlea and fertile valleya. FREE Irom swamps and milaria. and well 
watered by numerous running sireams. the banks of which are covered with 
GOOD TIMBER. 

4TH--Theio lands are FIRST CLASS AGRICULTURAL LANDS, adapted to 
raising Corn. Winter Wheat. Spring Wheat and all other Small grams, as well 
ii Broom Corn and Vegetables, thus admitting a rotation ol Crops. They are well 
watered, making oxcallent range for atock. 
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may rent some of it to other farmers for $100 per 40-acre 
unit. The game assumes you can always find someone to 
rent your available land or from whom you can rent. You 
need not try to rent from or to other participants. 

All costs, such as taxes, seed, labor, supplies, 
and maintenance are included in the charges for each of 
the farming activities. 

The Game Begins 

Now look at the activity sheet for 1880 that your 
instructor has given you. Fill in the blank for your 
background according to the farmer's role you are play- 
ing, the. year (1880), and the total amount of money you 
have ($1500) in the space next to "Capital." At the 
left is a list of all the crops and animals you may 
raise. The list indicates other possible expenses, 
such as cost of living, and has a space for a savings 
entry. Savings earn no interest. 

The first column, "Allocations," is the only col- 
umn you will fill in as you plan your farming activities. 
You fill in the other columns later. There is a sketch 
of your 160-acre farm on the right of the activity sheet. 
The costs for crops, livestock, and rent are also all 
listed on the activity sheet. As you decide how to use 
each 40-acrc square, write the decision or its abbrevia- 
tion on the map, calculate the expense, and write that 
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amount in the proper space in the "Allocations" column. 
For example, if you want to plant wheat in one 40-acre 
area, write "wheat" in one square and put $100 in the 
blank next to wheat in the "Allocations" column. Another 
sketch represents land you may rent. 

The total you may spend is the figure you nave re- 
corded as "Capital" at the top of the activity sheet. 
Continue making allocations until you have utilized all 
your land or capital. 

If, after you have made your allocations, you still 
have cash on hand but no land available, you may invest 
it in hogs, place it in savings, or rent more land. 

Figuring Outcomes 

Once you have made your allocations, you are ready 
to calculate how successful your farming has been. You 
will receive an outcome card. On it is a list of crops 
and livestock with a number or multiplier, printed after 
each commodity. If you planted 40 acres of corn and the 
outcome card shows a multiplier of 2 after corn, record 
the 2 in the "Multiplier" column on the activity sheet. 
Then multiply that number by the amount recorded in 
your "Allocations" column to figure the "Outcome," as 
indicated below. 

Allocation Multiplier Outcome 

Corn $200 2 $400 
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This operation should be repeated for each of the 
crops or animals you have chosen to raise. 

Notice that cost of living and rent are multi- 
plied by 0, and savings by 1. 

After you have calculated each commodity and 
entered the amount in the "Outcome" column, total this 
column. The difference between this total and $1500 is 
your net profit or loss for the year. The total of 
the "Outcome" column is the capital you have to work 
with in 1881. The amount should be entered on the 
1881 activity sheet. 

Be sure to read the introductory paragraph on your 
outcome card to find out why some of your investments 
failed and others were successful. You will have a 
chance to discuss your success with other participants. 

Now you are ready to make your allocations for 1881 
Base your decisions not only on your past experience but 
also on what you have learned during the first year's 
farming. After calculating your outcome, you go on 
to 1882. The procedure is the same. 

Playing the Game, 1920-21 

The game reopens in 1920. (Note that the 1919 peri 
od has been omitted) . Many changes have taken place 
since 1882. The changes that have the greatest impact 
on f irming in Western Kansas include the development of 
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farm equipment such as the tractor and the combine and 
the building of irrigation systems. New crops such as 
alfalfa, sorghum, and sugar beets have been introduced 
and adopted by farmers. 

Farmers have more knowledge about Western Kansas 
than the homesteaders had. Some of this information 
is summarized for you on graphs on pages 13 to 16 of 
this Manual. 

Because you have purchased an additional 160 acres, 
you must make mortgage payments. You have also pur- 
chased a tractor and other machinery. Your payments 
for indebtedness on land and machinery amount to 
$1000 per year. 

The prices you have been receiving for agricultural 
products are high, but your expenditures are also 
greater. Your living costs have risen to $2000 per 
year. You do have a greater amount of capital, how- 
ever, and you go into the 1920 period with $8500. 

Your farm now totals 32 0 acres and you must hire 
one laborer. His salary is $600 per year. 

Many of your neighbors in Settler County have expanded 
their operations by renting land. You, too, may rent land 
at $800 for 8 0 acres. This fee includes the cost of the 
additional ]abor necessary for expanding your operation. 

The activity sheet shows the cost for each crop 
or livestock. You have the option to plant unirrigated 
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wheat, barley, sorghum, or alfalfa, or irrigated 
sugar beets, alfall, or corn. 

If '.you decide t^ plant irrigated crops, you must 
pay an additional $200 the first year, which represents 
the capital costs of installing irrigation facilities. 
You may irrigate only 80 acres. 

Livestock choices include pasture (beef) cattle, 
dairy cattle, sheep, and hogs. There is an extra cost 
of $200 the first year you raise dairy cattle. Notice 
that you may raise up to $2400 worth of parture cattle, 
dairy cattle, or sheep on your 32 0 acres and that each 
$600 of investment, in livestock will require 80 acres 
of lana. Livestock may be mixed in units of $100. 
There is no limit to how much money you can invest in 
hogs . 

You are now ready to plan your strategy for play- 
ing the "Game of Farming" for the years 1920-21. 
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PART II: Till! OBJECTIVES OF THE GAME OF FARMING 



Now that you have played a part of the simula- 
tion game on farming, you|will spend a few minutes dis- 
cussing and analyzing it to determine what it teaches. 
After you have had the opportunity to list and discuss 
the objectives or learning outcomes of the game, you 
will see a video tape of an eleventh grade American 
history class playing the game. The taped session, which 
was completely unrehearsed, may indicate some learnings 
that you may not have considered before. As you watch 
the tape, ask yourself, "What are these students learn- 
ing from this simulation?" 

After viewing the video tape, you will be given 
a list of the objectives formulated by the game developers 
and will be asked to compare these with the objectives 
that your class has compiled. Discrepancies between 
the two lists are likely, for the class may have included 
objectives that the game developers have not listed, and 
vice versa. This should not disturb you. The game can 
be used to teach a wide range of objectives, and not all 
of them can be taught effectively at any one playing of 
the game. It is likely that the most important objectives 
will be included on both lists, however. 



PART III: EVALUATING THE OBJECTIVES 
Grading is so important in American education that 

m 

many people feel the purpose of testing or evaluation is 
only to determine grades. Actually, the purposes of 
evaluation are many and these different purposes re- 
quire a variety of testing techniques. 

In this part of the kit, you will analyze an 
evaluation instrument which utilizes several different 
evaluation techniques and which will yield data useful 
for several evaluation purposes. For this analysis 
you should look at the "Evaluation Instrument for the 
Game of Farming," on pages 20 to 25 of this Manual. 
You will also need the sheet entitled, "Some Objectives 
or Desired Learning Outcomes for the Game of Farming," 
which you were given in Part II of the kit. Since this 
evaluation instrument was devised by the developers of 
the game for the purpose of measuring its objectives, 
you should be able to match specific items on the instru 
ment with the stated objectives of the game. To do this 
set up two columns on a piece of paper, like this: 
Objective Number Test Item Number 

1 

2 
3 
4 
5 
6 
7 



As you examine each of the test items, try to 
match them to a specific objective. When you have 
finished matching the items with the objectives, you 
should read "Constructing and Interpreting an Evalua- 
tion Instrument," which begins on page 26 of this 
Manual. This reading outlines the procedures you would 
follow to develop and interpret the various components 
of a test like your sample evaluation instrument. 
Study the reading very carefully, for later you will 
be asked to construct your own items and scales. 
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EVALUATION INSTRUMENT FOR THE GAME OF FARMING 



Directions: Each of the questions or incomplete statements in Part I is 

followed by four suggested answers or completions. Select 
the one which is best in each case. 



Part I 

1. Which of. the following. played. the most important role in luring 
settlers;. to..Western Kansas in the decades after the Civil War? 

*A. Advertising by the railroad companies 

B. The dis.covery of gold near Denver 

C. Advertising by the Cattlemen's Associations and Homesteader's Union 

D. The virtual disappearance of hostile Indians in the area 

2. If a settler in Eastern Colorado in the 1880 «s was deciding whether to 
grow wheat or corn, which of the following factors should play the most 
important role in his decision? 

A. The labor requirements of the crops 

*B. The climatic requirements of the crops 

C. The availability of appropriate machinery 

D. The availability of livestock 

3. Which of the factors listed as options for question 2 (A, B, C or D) 
would play a less sig nificant role in a Colorado farmer's decision whether 
to grow wheat or corn during World War I? 

A. The labor requirements of the crops 

*B. The climatic requirements of the crops 

C. The availability of appropriate machinery 

D. The availability of livestock 

4. In a paragraph of no more than 50 words explain your answer to ques- 
tion three. 



9 

ERIC 



21 

5. Which of the following is an example of an event which could tend 
to cause a rise in the market price of corn? 

A. A report of the prospect of a bumper crop throughout the U.S. 

B. The report of excellent prospects for a large crop in Europe 

C. A report that a much needed rain had fallen in the corn belt 
*D. A report that an early frost had occurred in several sections of 

the corn belt 

6. Which of the following acts or doctrines was most influential in 
the rapid settlement of the American West? 

A. Embargo Act 

B. Freeport Doctrine 
*C. Homestead Act 

D. Specie Circular 

7. Which of the following would be the most likely outcome of a grain 
farmer's decision to irrigate his land? 

A. An increase in the price he can obtain for each bushel 

B. A decrease in his production costs 
*C. An increase in his yield per acre 

D. An increase of his business risks 

8. Three of the following statements about agriculture and manufacturing 
are correct. Which one is NOT correct? 

A. The farmer has less control over the quality of what he produces 
than the manufacturer. 
*B. The farmer usually requires more capital for machinery than the 

manufacturer. 

C. Farm work is more seasonal than manufacturing. 

D. Farming requires greater amounts of space than manufacturing. 
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Corn yields, bushels per acre 
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Corn, price per bushel in dollars 
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9. The data for the period from 1915-1920 on the above graphs indicate 
that price and production rates: 
A. vary together 
*B. are inversely variable 

C. are unrelated 

D. do not vary 



Part II 



In Part II, you are asked for your opinions. Questions 10 through 
14 ask you to rate the following topics in terms of how interesting you 
think they would be to study. 



A. dull 

H. generally uninteresting 

C. generally interesting 

I). very interesting 



10. Manufacturing 

11. Settlement of the American 
West 

12. Politics 

13. Farming 

14. Kansas 
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Part III 



Directions: In Part III we want to find out how you describe different 

things. As in Part II, there are no right and wrong 
answers. You will find a word printed like this: 

FARMING 

Look at the word; get an idea (image) of it in your own mind. Below 
the word you will find a number of words which describe farming. These, 
words are put in pairs that have opposite meaning, like this: 



Good Bad 

Hard " Eas y 



Between the words with opposite meaning are seven spaces. You are asked 
to fill in the space that you feel best describes farming. For example, 
if you feel farming is neither "good" nor "bad", you would fill in the 
space in the middle, like this: 

Good W Bad 



If you feel that the meaning of farming is a little more good than bad, 
you would put your mark like this: 



Good 



Bad 



If you feel farming is very good, then, you would mark like this: 
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FARMERS 



good ~ bad 

^ 

* smart ~ ~ ZZ — — — stupid 

hard ~ ~ ZZ eas y 

serious ZZ ~ ZZ ZZ ZZ ZZ ZZ funn y 

important ~ '~ " ~ ZZ unimportant 

proud - ~'\Z2~- - Zll r * " ashamed*" 

happy ^3 ZZ ZZ ZZ saci 

clean ~ ~ ZZ ZZ ZZ ZZ ZZ dirt y 

educated ZZ ZZ ZZ ZZ ZZ ZZ ZZ uneducated 

interesting ZZ ZZ ZZ ZZ ZZ ZZ ZZ uninteresting 

I* GEOGRAPHY 

logical ZZ ZZ ZZ ZZ ZZ ZZ ZZ illogical 

"meaningful ZZ ZZ ZZ ZZ ZZ ZZ ZZ meaningless 

hard ZZ ZZ ZZ ZZ ZZ ZZ ZZ eas y 

clear ZZ ZZ ZZ ZZ ZZ ZZ ZZ unclear 

important ZZ ZZ ZZ ZZ ZZ ZZ ZZ unimportant 

useful ZZ ZZ ZZ ZZ ZZ ZZ ZZ useless 

analytical ~ ZZ ZZ ZZ ZZ ZZ ZZ descriptive 

... r , not important fo 

X r fu n ture°' — m — the 1 future 

interesting ZZ ZZ ZZ ZZ ZZ ZZ ZZ uninteresting 
precise ZZ ZZ ZZ ZZ ZZ ZZ ZZ va 8 uc 



9 

ERIC 



25 



Part IV 

# 

Directions: Each of the following statements is followed by five 

• t 

responses. Indicate how you feel about each statement 
by circling one of the responses. 

Circle the SA if you "strongly agree" with the statement 

Circle the A if you simply "agree" with the statement 

Circle the UN if you are "uncertain" of your feelings 
about the statement 

Circle the DA if you "disagree" with the statement 

Circle the SPA if you "strongly disagree" with the 
statement 

Here, also, there are no right answers; only your 
opinions are sought. 



17. Farming does not require much 
• education or intelligence 



SA A UN DA SDA 



18. Farming is probably an easier 
way to earn a living than most 
occupations. 



SA A UN DA SDA 



19. FarmingMs risky business 



SA A UN DA SDA 



20. It takes a lot of courage to 
go into farming. 



SA A UN DA SDA 
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Constructing and Interpreting an Evaluation Instrument 

The Evaluation Instrument for the M Game of Farming" 
contains four parts which illustrate one or more testing 
techniques. 

Part I. Objective and essay questions measuring 
knowledge objectives 

Part II. An interest scaxe measuring interest in 
subjects of study 

Part III. A semantic differential scale measuring 
attitudes 

Part IV. A Likert Scale measuring attitude-.: 
Each of these techniques will yield data useful to the 
evaluation conscious teacher. Part I, dealing with 
knowledge objectives, yields information that will pro- 
vide a tangible basis for assigning student grades. It 
will also provide information about how well the class 
as a group has performed on the knowledge or intellec- 
tual skill objectives for a particular unit or course. 
Parts II, III, and IV, while not providing information 
that is gradable, will yield data about group perfor- 
mance on atcitudinal or appreciation objectives. 

The following is a collection of essays on using 
these techniques. As you read them, you^will want to 
refer back to the part of the evaluation instrument . 
being described. • • 
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Objective and Essay Questions 

Tests which measure knowledge, skills, and under- 
standing's have been convent ionalxy divided into objec- 
tive and essay categories. If the score on the test 
remains the same, regardless of who does the scoring, 
the test is considered obj :tive. If the score differs 
with different scorers, the test is considered an essay 
or subjective test. Multiple choice questions like 
questions 1, 2, and 3 of Part I are, of course, examples 
of objective questions, while question 4 is an example 
of an essay or subjective question. 

These types of questions have advantages and dis- 
advantages. For instance, essay tests, though easily 
prepared, suffer from low scorer reliability, which means 
that the grades for the same essay will differ greatly 
from teacher to teacher and for the same teacher on 
two different evaluations. This problem is well illus- 
trated by a recent research study. In the study 53 
instructors were asked independently to rate 300 essays 
written by college freshmen on a nine point rating scale. 
When the graded essays were returned, 34 per cent of 
the ratings had received all nine ratings, 71 per cent 
of the essays had a range of eight ratings or more, 
94 per cent had a range of seven or more, and 99 per cent 
had a range of six ratings or more. With essay tests, 
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the grade assigned largely depends on who is makr 7 the 
evaluation rather than on what is being evaluated. 

With essay tests, too, the verbal ability of the 
student is often evaluated rather than his substantive 
knowledge. Meager knowledge can be embellished and con- 
sequently overrated. 

Objective tests have substantially better scprej 
reliability than do essay tests, and they are not subject 
to contamination by writing or penmanship abilities. 
However, objective tests are difficult to construct and, 
if poorly constructed, may emphasize insignificant fac- 
tual detail. 

Constructing Essay Items . Select the questions with 
care. Since there can be only a few questions, those 
which deal with the most central and important topics 
should be used. When you have chosen your topics, be 
explicit in your instructions sc that all examinees 
address themselves to the intended question. Some 
11 1 cs t - wise 11 students will write as if they have mis- 
understood the directions to disguise their weakness 
on a topic . Words 1 ike "compare 11 and "contrast 11 
are preferable in the instructions to terms such as 
"discuss," "write an essay on," e*c. 

As a rule, do not offer a choice of questions. 
This greatly reduces the already limited opportunity 
for assessing individual differences and, at the same 
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time, makes comparative scores more difficult to inter- 
pret. 

It is a good idea to indicate the approximate length 
of the essay you expect the student to write and the 
credit that will be assigned to the essay relative to 
other portions of the test. Make sure that the exam- 
ination is not too long for the allotted time. Ample 
time should be provided for the examinee to give some 
thought and planning to his answer as well as to the 
written effort. 

Grading Essay Items . Prepare a model answer, out- 
lining the main points to be covered in each question. 
The credit given for each item, as well as for each 
main point, should be decided in advance. When grading, 
you should read the examinations anonymously. If the 
examinee is known, the "halo" effect will consciously 
or unconsciously contaminate the ratings. The fact that 
a student is well-behaved (or ill-behaved) should not 
be spuriously "read into" test scores. 

To avoid the "serial" effect, quickly read the 
first question on all papers and place each into one 
of three (or five) categories from poor to excellent. 
Otherwise, an exam following two or three excellent 
papers is underrated due to the serial effect. Like- 
wise, an average paper may appear excellent if preceded 
by several poor efforts. 
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Grade the answers to the first question on all 
papers, assigning a point value, before going to the 
second. question. If this is not done, the "halo" 
from the previous items will color the evaluation on 
the subsequent items. The student's effort on question 
two should be evaluated independently of his performance 

on question one. 

Constructing Objective Items . Begin by formulating 
a "blueprint" or plan in order that the various aspects 
of the content are weighted proportionately. Proportion- 
al weighting will not occur if you write items randomly 
until you have the desired number of questions. 

"fry' to keep items "criterion -or ien ted. " Ask your- 
self, "What are the important concepts of this unit; 
what are my objectives?" This step will help you to 
avoid trivial "so what" items. Also try not to use 
the stereotyped phrasing of the textbook. Use new 
text, graph, or map materials to reconstruct the meaning 
in a fresh setting. This will raise the level of under- 
standing necessary to be credited with the item and 
will "unconait ion" the pupils away from memorization 
without understanding. 

Make the test items easy to read unless the measure- 
ment of reading ability is your objective. This rule- 
is especially important at the elementary school level. 
If an assessment of arithmetic reasoning is desired, the 
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assessment should not be contaminated by measuring 
reading ability concurrently. 

Allow the position of the correct answer to vary 
randomly over the various positions. A teacher whose 
true-false items are false most of the time will find 
that his students will be able to guess with greater 
than chance success. The same is true if a test is keyed 
in some other non-chance fashion. For example, if an 
expression such as "none of the above," "more than one 
of the above," "all of the above," or "both A and B," 
etc., is consistently keyed correct or incorrect, an^ ^ 
irrelevant clue is given to the "test-wise" student. 

Finally, bee . ;>e of the severe limitations on 
teachers' time, you should probably never allow an 
examination to leave the classroom even after it has 
been given. Often, it is not practical for a teacher 
to compose an entirely new and welt-designed test for 
each grading period or semester. On the other hand, 
an examination should not be repeatedly used without 
change, but should be kept current by periodic updating. 

* * * 

An Interest Measurement Scale 

The interest measurement technique exemplified in 
Part 11 of the "Evaluation Instrument for the Game of 
Farming" was devised by curriculum developers to measure 
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student interest in school and in the various subjects 
studied. Like the other attitude scales used in the 
instrument, it is a relatively unrefined technique. 
Most attitude scales have recognized limitations in 
terms of reliability when used to gain insights about 
the attitudes of individuals. They can, however, be 
invaluable tools for teachers who wish to measure the 
impact of their teaching on the attitudes of their 
class as a group. 

Constructing an Interest Measurement Scale . The 
interest measurement scale is a simple procedure. The 
evaluator establishes four options, A, B, C, and D, 
which 'represent points on an interest continuum. 

A. dull 

B. generally uninteresting 

C. generally interesting 

D. very interesting 

He then lists the topics or ideas tnat are to be 
studied and asks the respondents to react to each topic 
in terms of how interesting he thinks it is. For 
example, "Rate the following topics in terms of how 
interesting you think they would be to study." 
1. The Civil War 

2. The period of reconstruction 

3. The settlement of the American West 
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To have some points of comparison in assessing the 
degree of interest your students have registered, you 
will probably want to include topics other than those 
actually to be studied. This, of course, is the reason 
for the inclusion of items 10, 12, and 14 in Part II of 
the evaluation instrument. "Sports" is a good compari- 
son topic to include in an interest scale. Students 
will usually rate sports as highly interesting. Any 
course topic which was rated as high as "sports" at 
the conclusion of a unit of study could probably be 
interpreted as having been very successful in stimu- 
lating student interest. 

Interpreting the Interest Scale . The interest 
scale, like the other attitudinal measures that will 
be studied, should be administered before and after the 
unit you wish to evaluate. In each administration you 
will have your students register the degree of interest 
they have for each topic listed. To scor each topic, 
tabulate the number of A's, B's, C's, and D's registered. 
Then assign a weighted value to each of the four options. 
The "very interesting" response receives a 4 value, and 
so on to the "dull" response which would receive a value 
of 1. You determine a class mean for each topic by 
totaling the weighted values and dividing by the number 
of students responding. 

The following data was collected from the administration 
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of this interest scale before and after a geography unit 
on manufacturing and agriculture was taught. 

Class Mean' Class Mean 
Topic for Pretest for Posttest 

1) Politics 2. 62 2. 65 

2) Farming 2.02 * 3.05 

3) Japan 2. 70 2. 69 

4) Manufacturing 2.33 2.57 

Topics 1 and 3 were not a part of the unit of study 
and showed little, if any, gain in student interest. 
Topics 2 and 4 were emphasized in the unit and did show 
substantia] gains. There is some additional valuable 
feedback in relation to topics 2 and 4. Both of these 
topics received equal class time and yet topic 2 gener- 
ated considerably more interest. The teacher who wanted 
a high student interest in the subject he is teaching 
would use this evaluation data by attempting to replicate 
the procedures or materials employed in teaching topic 
2 for future topics of study. 



* * * 



The Semantic Differential 

The semantic differential technique, which is ex- 
emplified in Part III of the evaluation instrument, is 
another attitude measurement scale. The -technique was 
developed by Charles K. Osgood as a part of a research 
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study of meaning. The instrument has been used to ex- 
plore the images or meanings that people have of ideas 
or things, from brands of cigarettes to socialism. For 
whatever is being rated, the evaluator selects a number 
of bipolar adjectives or adjective phrases which will 
define the dimensions of meaning. For example*, in 
determining the image that people have of a particular 
brand of cigarette, a researcher might choose: mild- 
strong, cool-hot, masculine-feminine, a quick smoke- 
a long smoke. These adjectives are separated by a seven 
point scale on which the respondent registers his atti- 
tude toward the brand of cigarettes. Care must be taken 
that the two adjectives at the extremes really are op- 
posed and do define some kind ^Tf continuum or dimension 

between them. 

Constructing a Semantic Differential . Researchers 
randomly assign the location of the positive end of 
the continuum, so that in the case of the cigarette 
research "mild," "cool," and "a long smoke" would 
probably not fall on the same side. For simplicity in 
organizing a semantic differential for yourself, however 
you may not want to be so precise. Scoring and intcrpre 
ting will be easier if you arrange your pairs of adjec- 
tives so that any change taking place from the first 
administration of the instrument to the second will 
move from the right to the left of each continuum. For 
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instance, if you are interested in "how smart," "how 
important," and "how educated" students think farmers 
are, and want the lessons that you had prepared to move 
the students to more favorable attitudes on all three 
continua, you would put "smart," "important," and "edu- 
cated" on the left-hand column. "Stupid," "unimportant," 
and "uneducated" would be placed in the right-hand column. 

You may include other pairs of adjectives that are 
not of real interest to you just to veil your intent. 
In the measurement of a class's conception of farmers, 
the "happy-sad," "serious-funny" continua would be 
examples of those that would probably not yield useful 
information. It would not matter, of course, how these 
adjectives were placed. 

Interpreting a Semantic Differential . When your 
students have registered their attitude toward farmers 
or geography, or whatever the subject you want to evaluate, 
you will want to determine a class average for each 
continuum of interest to you. Assume you are working 
with the "unimportant- import ant" continuum. You would 
begin by assigning a value from 1 to 7 for each of the 
seven spaces between these adjectives. The student 
markings in the space to the far right would carry the 
value of 1 and those on the far left a 7. Each space 
in between carries an appropriate value between these 
extremes. First, total the number of studen'ts who have 
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marked space 1 and multiply by one, then total the num- 
ber of 2's and multiply by two, and so on through the 
seven spaces. Then divide by the total number of stu- 
dents and you will have a weighted average for your 
class. As an example: 

FARMERS 

7 6 5 4" 3 l 1 
important unimportant 

Suppose your class of 20 students reacted to the above 
continuum in the following way: 

2 students marked space 1 giving that space a total value of 1 



3 ii ti ii 2 " " " " " " " 6 

6 ,J - » "3 " " " " " " " 18 

6 " " " 4 " " " " " " " 24 

0 " " " 5 " M " " " " " 0 

3 " " " 6 " " " " " " " 18 

0 " M " 7 11 " " " " " " 0 



TOTAL 68 



The total (68) divided by the number of students (20) gives you 
a mean value for this continuum of 3.40. When you have 
determined a mean value like this for each continuum of 
interest to you, a class profile for this concept can 
be drawn. 

If you had administered this same instrument before 
and after an instructional unit, you might obtain pro- 
files like those oil the following page. They indicate 
information about changes in the attitudes of your class 
as a result of t ho lessons you hrd prepared. 

erJc 



4*. -» * 



38 



Farmers 



good 



smart 



hard 



s.er ious 



important 



proud 



happy 



clean 



educated 



interesting 




bad 
stupid 
easy 
funny 

un important 
ashamed 
sad 
dirty 

uneducated 
uninteresting 



pretest class profile - 
posttest class profile 



* * * 



The Likert Scale 



C on s t rue ting a I, i k c r t Scj-lc . The measurement device 
exemplified in Part IV of the evaluation instrument was 
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developed by Rensis Likert. It is probably the most 
easily constructed and widely used scale presently 
available for measuring attitudes. With this device 
the evaluator formulates statements that would reveal 
attitudes that are relevant to the topic being studied. 
For example, if race relations was being studied and 
you wanted to evaluate your class's attitude about 
this topic, you might devise statements similar to these: 

1. I would welcome Negroes in my place of business. 

2. Chinese should not be allowed in our country. 

3. Indians are born criminals. 

The respondents must react to the statement by marking 
a five point continuum, from "strongly agree," to "agree, 
"uncertain," "disagree," and "strongly disagree." 

Write these attitude statements so that they appear 
meaningful and interesting, even exciting, to the student 
When the Likert technique has been used unsuccessfully, 
it is usually because the statements have failed to 
arouse much interest in the respondents. If you are 
producing appropriate attitude statements, your 
respondents probably will not quibble or want to change 
statement.. Also, large numbers of "uncertain" re- 
sponses or statements skipped by students should indi- 
cate that statements are not functioning well. 

Intera cting a L ikcjt__Scalc . After your students 
have registered their reaction to each statement on your 



scale, you will determine a class mean for each state- 
ment. This is done similarly to the method employed 
with the semantic differential and other attitudinal 
scales already discussed. Each of the five response 
positions is assigned a weight of 5, 4, 3, 2, or 1. If 
you decide that a high score on the item or statement 
would indicate a favorable attitude, then favorable 
statements roust be scored 5 for "strongly agree," 
down to 1 for "strongly disagree." Likewise, un- 
favorable statements must be scored 1 for "strongly 
agree" up to 5 for "strongly disagree." 

Some examples may be helpful here. If you were 
teaching a unit on United States foreign policy and 
wanted to evaluate your students' attitudes toward 
this policy, you might develop a scale to do so that 
included the following items. 

1. The U.S. should drop its tariffs on products 
coming from Latin America. SA A U DA SDA 

2. The U.S. should terminate all military alliance 
agreements with foreign nations. SA A U DA SDA 

If you decided that a high score on statement 1 
would indicate a favorable attitude, you would record 
a 5 for every student who registered a "stiongly agree," 
moving down to recording a 1 for every student who 
strongly disagreed. A class mean for the statement would 
be determined by adding these weightings and dividing by 



the total number of students in the class. 

Similarly, you might docide that statement 2 
represented an unfavorable statement, If this were so, 
you would record a 1 for every student who registered 
a "strongly agree," up to a 5 for every student marking 
"strongly disagree." The class mean would, of course, 
bo determined in the same way as for statement 1. 

There is no easy way to graph the results of the 
Likert Scale as there is for the semantic differential. 
Changes that might take place in the attitudes of your 
group from pre-unit to post-unit evaluations, however, 
can be clearly demonstrated by changes in the class 
means that are obtained. 



42 



PART IV: USING FEEDBACK EVALUATION 



■ 

In this part of the kit you will work a programmed 
exorcise to instruct yourself about the uses of evalua- 
tion data as feedback to improve teaching. When you have 
worked the exercise, your instructor will lead a class 
discussion about it. 

The materials needed for this part include: the 
program and its answer sheet, a handout sheet, "Some 
Objectives or Desired Learning Outcomes for the Game of 
Farming," and the "Evaluation Instrument for the Game 
of Farming," on pages 20 to 25 of this Manual. 

A programmed learning exercise is not like a test 
although it might resemble a test. A program is a 
method of presenting information, like a reading inter- 
spersed with questions that you must answer. Your 
answers indicate that you underst?nd the material being 
presented. Although you have an answer sheet on which 
to record your responses to the questions, only you 
will look at the sheet. The answer sheet serves to 
point out the questions that you have missed so that you 
are able to correct your errors. More specific direc- 
tions for the program are included with it. 
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PART V: APPLICATION 

During the final two hours of work on this kit, 
you will have an opportunity to apply some of the tech- 
niques and ideas that you worked with in previous parts 
of the kit. You will work with a list of objectives 
for another classroom activity, "Portsville , " This list 
is found on pages 44-46 of this Manual. If you partici- 
pated in the kit dealing with simulation, the "Portsville" 
activity will be familiar to you. ("Portsville" is a 
classroom activity in the High School Geography Project's 
unit, Geography of Cities , published by The Macmillan 
Company.) After you have read the objectives, you view 
a video tape of students participating in the activity 
to see how the objectives are actually taught. Using 
this background, you will devise items and scales to 
measure che attainment of the activity's objectives. 

Some materials are provided from which you can gen- 
erate test items to measure the knowledge or intellec- 
tual skill objectives. A sketch map of the hypothetical 
community "Holly Hay" is found on page 47 of this Manual. 
Also, your instructor will hand out the "New Orleans 
liast" topographic maps that you worked with in another 
HSGP kit. You may remember from the reading "Constructing 
and Interpreting an Evaluation Instrument," assigned in 
Part III, that il is a good idea to structure questions 



in a "fresh setting" when possible. For example, if 
students have been taught the use of map scale or the 
principles governing land use relationships on a map of 
"Portsvillc" (Seattle, Washington, in reality), they 
should bo asked to demonstrate this learning on some 
other map. This transfer raises the level. of under- 
standing necessary to answer the item correctly and also 
serves to condition students away from memorizing with- 
out understanding. 

Educational Objectives for "Portsvillc" 

In addition to the more specific knowledge objec- 
tives listed below, some social and attitudinal objec- 
tives arc very important. Objectives one and two are 
most important among these. 

1. The student completing this activity will be 
better able to participate in making group decisions. 
Working in a group to build a model of a city should 
improve a student's ability to function effectively in 
other group decis ion-maKing situations. 

2. At the conclusion of this activity, the student 1 
attitude toward the city will have been modified: 

a. lie will find the city a more interesting 
topic to study. 

b. No longer overwhelmed by the city's size 



and complexity, he should be able to see the 
city as a comprehensible whole, an assemblage 
of rational parts, 
At the conclusion of the activity, the student will 

be improved in certain knowledge and intellectual skills. 

He should be better able to: 

3. Predict where different kinds of urban land use 
are likely to be located. For example, given a detailed 
map of an urban area and asked to predict the most prob- 
able locations for different types of land use, the stu- 
dent locates heavy industry along water and rail routes, 
shopping and recreational centers adjacent to residen- 
tial areas, and high cost single- family dwellings on the 
outskirts of the city. 

4. Explain why different kinds of urban land use 
may be located where they are. For example, when asked 
to explain why heavy industry is located near water or 
rail transportation, the student refers to the need of 
such industry to be accessible to both raw materials and 
markets. 

5. Explain why a city grows or declines. For 
example, told that a city of 500,000 now prospers where 
a small settlement once existed and asked what informa- 
tion he would need to account for this growth, he men- 
tions information about the transportation connections 
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with other areas, the goods and services produced by 

the city, the pattern of population growth in the overall 

area, and the availability of natural resources. 

6. Read and interpret the scale of a map. For 
example, given a map with its scale on it, and asked to 
indicate which shopping centers would be conveniently 
accessible to a person walking from any given point in 
the city, the student does so. 

7. Cite examples showing how people have modified 
their physical environment and adapted to it. For ex- 
ample, given a large scale map of New Orleans and as! ed 
to cite specific examples where tho people h*ve either 
modified their physical environment or adapted to it, 
the student points out several examples. 
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